The Ethical Challenges of Artificial Intelligence (AI) Algorithms in Health Care

Generative artificial intelligence (AI), a groundbreaking health care technology in the United States, could create ethical challenges without proper safeguards. Of particular concern are AI clinical algorithms, which critics claims could magnify harmful biases and exacerbate health disparities. Since initiation of three class action lawsuits in 2023, alleging AI insurance coverage algorithms denied medically necessary care via an almost fully automated claims decision process, largely devoid of human oversight, such has become a public concern. Our case illustration details major flaws in application of such a coverage algorithm, as an elderly Medicare Advantage Plan beneficiary is left without access to essential medical care. Although advocates at the state and national level are seeking remediation, such practices will likely persist, as insurers know only a small percentage will appeal these denials. It is apparent these algorithms are not a “crystal ball for care,” as human input is needed at critical junctures in the process. The design, development and deployment of these AI tools must be done in the utmost ethically responsible way, similar to what some health entities have already accomplished. Such perhaps can be best accomplished by adhering to the ethical principles established by the World Health Organization. By doing so, key stakeholders can help assure that these generative AI clinical support tools are utilized as support tools, and do not replace the clinical team while preserving human oversight.